Adaptive cyber-physical system for efficient monitoring of unstructured environments

ABSTRACT

The present disclosure provides a system for monitoring unstructured environments. A predetermined path can be determined according to an assignment of geolocations to one or more agronomically anomalous target areas, where the one or more agronomically anomalous target areas are determined according to an analysis of a plurality of first images that automatically identifies a target area that deviates from a determination of an average of the plurality of first images that represents an anomalous place within a predetermined area, where the plurality of first images of the predetermined area are captured by a camera during a flight over the predetermined area. A camera of an unmanned vehicle can capture at least one second image of the one or more agronomically anomalous target areas as the unmanned vehicle travels along the predetermined path.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/480,076, filed on Jan. 22, 2018, which is a U.S. National Stage ofInternational Application No. PCT/US18/14695, having an InternationalFiling Date of Jan. 22, 2018, which claims the benefit of U.S.Provisional Application No. 62/449,439, filed Jan. 23, 2017. Allsections of the aforementioned application(s) and/or patent(s) areincorporated herein by reference in their entirety.

GOVERNMENT LICENSE RIGHTS

This invention was made with government support under 1720695 awarded bythe National Science Foundation. The government has certain rights inthe invention.

FIELD OF THE DISCLOSURE

The subject disclosure relates to a method and system for monitoring ofunstructured environments.

BACKGROUND

Improving food security, while at the same time enhancing theenvironmental sustainability and economic viability of large-scaleproduction agriculture is a momentous challenge. Intensive fieldmonitoring is a highly effective way of boosting yields, since it allowsgrowers to identify potential problems such as weeds, diseases, pestsand nutrient deficiencies early, and correct them reliably andcost-effectively. However, the shortage of well-trained agronomists, aswell as the prohibitive costs of intensively monitoring fields inperson, results in most fields being monitored less frequently and lessthoroughly than desirable. Therefore, there is an urgent need forinnovative technical solutions that improve the actionability,ease-of-use, cost-effectiveness, and labor-efficiency of in-seasonmonitoring of fields producing commodity crops like soybean.

Current aerial-monitoring products do not deliver readily actionableinformation despite requiring high cost sensors and high-bandwidthcloud-connectivity, and being challenging to use. Even after the typicalturn-around times of 8-24 hours to deliver analyses such as NormalizedDifference Vegetation Index (NDVI), they do not deliver specificactionable information directly. Instead, current analyses serve only toindicate potential problem areas in a field and require the agronomistto re-visit the field and visually inspect these areas to verify theproblem and ascertain its cause. Therefore, there is a strong desire fortechnological solutions that actually improve the productivity ofin-season field monitoring, and for other applications includingranching, forestry, natural environments and infrastructure monitoring.

BRIEF DESCRIPTION OF THE DRAWINGS

The features, objects and advantages other than those set forth abovewill become more readily apparent when consideration is given to thedetailed description below. Such detailed description makes reference tothe following drawings, wherein:

FIG. 1 illustrates workflow of a field-monitoring method that providesfarmers with close-up pictures of anomalous areas of the farm, withouthaving to wait for cloud based analysis. Feedback from users on thedelivered images can be used to improve the algorithms. The workflow canbe applied for monitoring of unstructured environments.

FIG. 2 illustrates a yield-gap between award winning soybean yields andthe state-wide average yields of irrigated soybean in 2012 in the topten soybean producing states. The average unrealized yield of about 40bushels/acre amounts to around $125,000 for an average farm operation(400 acres) at current soybean prices.

FIG. 3 illustrates an image analysis from an adaptive flight-planningmethod which enables on-site detection of anomalous areas. A highaltitude RGB image from a Unmanned Aerial or Aircraft System or Vehicle(UAS) (left image), is transformed to an HSV (hue-saturation-value)colorspace on which Deviance Detection algorithm can find anomalousareas (center image). A closeup of these areas is obtained by the UAS inthe second stage of the same mission (right image). The application canautomate this workflow and rapidly deliver actionable information, suchas without the need for a cloud connection.

FIG. 4 illustrates detection by the exemplary method of areas of highagronomic interest in RGB images of a field (left image) with clustering(center image) and exemplary deviance detection (right image)algorithms. The methodology implements these algorithms in a robust,reliable, fast, and power-efficient manner, such as on tablet computers.The algorithms can be validated on a broad variety of crop-issues,optimizing on the above criteria, and improving their robustness toirrelevant signals (such as clouds or non-agricultural objects).

FIGS. 5A-5B illustrate: 5A. (left image) shows clusters found in animage at 20% of original resolution. 5B. (right image) shows clustersfound with an image at 100% resolution.

FIG. 5C illustrates computational speed vs image resolution for theexemplary method.

FIG. 6 depicts results of the exemplary method on sample data from asoybean farm near Ogden Ill. The white trace shows the stage-1 gridpath. A total of 26 interesting areas were found, their locations andassociated figures are shown on the path. The red trace shows thealgorithmically planned stage-2 path where the UAS visits 10 of theinteresting areas to obtain close-up images. This methodology can beimplemented on mobile devices and can result in an optimized stage-2mission plan.

FIGS. 7A and 7B illustrate distortion removal performed by the exemplarymethod. In FIG. 7A, the original raw image has significant distortionaround the periphery. In FIG. 7B, the undistorted image is generatedwhere the rows become properly straightened after correction.

FIGS. 8A-8B illustrate clustered images. A DP-means algorithm is appliedto find two clusters within the image, where A) cluster 1 corresponds tothe background and B) cluster 2 is an anomaly.

FIG. 9 illustrates the exemplary methodology being applied and resultingin the identified top five Regions of Interest (ROIs) from the anomalousblobs.

FIG. 10 illustrates a Visualization of Genetic Solver Process that canbe used as part of the exemplary method.

FIG. 11 illustrates Percent Quality Difference of Distance of FlightPath between TSP Genetic Solver and Nearest Neighbor Search For 10, 20,50 Uniformly Distributed Points.

FIG. 12 illustrates Percent Quality Difference of Travel Time betweenTSP Genetic Solver and Nearest Neighbor Search for 10, 20, and 50Uniformly Distributed Points.

FIG. 13 illustrates Computation time in seconds of the Genetic Solverfor the Traveling Salesman Problem (TSP) for 10, 20, and 50 UniformlyDistributed Points.

FIG. 14 illustrates Computation time in seconds of the Nearest NeighborAlgorithm for 10, 20, and 50 Uniformly Distributed Points.

FIG. 15 illustrates Computation time for each combination of populationsize and number of points with only the minimum iterations needed areused for each respective rep.

FIG. 16 illustrates Minimum Iterations necessary for solver to convergeon a solution for each combination of Population Sizes and Number ofPoints.

FIG. 17 illustrates Percent Quality Difference in Travel Time betweendifferent combinations of Gene population size and the number of points.

FIG. 18 illustrates Percent Quality Difference in Travel Time betweendifferent combinations of Gene population size and the number of pointswith only the minimum amount of iterations for each respective rep.

FIG. 19 illustrates Computation time for the Nearest Neighbor SearchAlgorithm for points distributed along Uniform and Gaussiandistributions in the Second Comparison Analysis.

FIG. 20 illustrates Computation time for the Genetic Solver of theTraveling Salesman Problem for points distributed along Uniform andGaussian distributions in the Second Comparison Analysis.

FIG. 21 illustrates Percent Quality Difference between the Travel Timeof the routes from the Genetic Solver and the Nearest Neighbor Search inthe Second Comparison Analysis.

FIG. 22 illustrates a workflow showing the user interface of anexemplary embodiment for monitoring of unstructured environments.

FIG. 23 illustrates a workflow showing ability of a user to providefeedback of an exemplary embodiment for monitoring of unstructuredenvironments.

FIG. 24 illustrates a comparison between a commercial service workflowand a workflow showing ability of a user to provide feedback of anexemplary embodiment for monitoring of unstructured environments.

FIG. 25 is a diagrammatic representation of a machine in the form of acomputer system within which a set of instructions, when executed, maycause the machine to perform any one or more of the methods describedherein.

FIGS. 26A, B, and C illustrate the first and second flight paths for anunmanned aerial vehicle according to an embodiment for afield-monitoring method including an overview of the predetermined areaand images of detected anomalous areas.

While the present invention is susceptible to various modifications andalternative forms, exemplary embodiments thereof are shown by way ofexample in the drawings and are herein described in detail. It should beunderstood, however, that the description of exemplary embodiments isnot intended to limit the invention to the particular forms disclosed,but on the contrary, the intention is to cover all modifications,equivalents and alternatives falling within the spirit and scope of theinvention as defined by the embodiments above and the claims below.Reference should therefore be made to the embodiments above and claimsbelow for interpreting the scope of the invention.

DETAILED DESCRIPTION

The system and methods now will be described more fully hereinafter withreference to the accompanying drawings, in which some, but not allembodiments of the invention are shown. Indeed, the invention may beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein; rather, these embodiments areprovided so that this disclosure will satisfy applicable legalrequirements.

Likewise, many modifications and other embodiments of the system andmethods described herein will come to mind to one of skill in the art towhich the invention pertains having the benefit of the teachingspresented in the foregoing descriptions and the associated drawings.Therefore, it is to be understood that the invention is not to belimited to the specific embodiments disclosed and that modifications andother embodiments are intended to be included within the scope of theappended claims. Although specific terms are employed herein, they areused in a generic and descriptive sense only and not for purposes oflimitation.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of skill in the artto which the invention pertains. Although any methods and materialssimilar to or equivalent to those described herein can be used in thepractice or testing of the present invention, the preferred methods andmaterials are described herein.

Overview

The two key hurdles to realizing the potential benefits of UAS or otherunmanned vehicles for monitoring of unstructured environments (e.g., inagriculture) that are addressed by the present disclosure are highcapital and labor costs of using drones and the impracticality of use inrural areas. By way of summary, the present disclosure provides anadaptive cyber-physical system for efficient monitoring of unstructuredenvironments using adaptive multi-stage flight planning algorithms forefficient and informative field monitoring to increase monitoringefficiency and reduce labor costs. In addition, computationallyefficient algorithms were created for in-situ data analysis obviatingthe need for uploading large amounts of data to the cloud for analysis,which can be a substantial hurdle in areas with poor internetconnectivity. Also, algorithms for identifying and classifying potentialinformative areas from the field or area of interest have been createdto improve continually by learning from user feedback. Theseinnovations, motivated by real agronomic needs, are essential forcreating compelling products and a novel approach. An example of cropyields is illustrated in FIG. 2.

Disclosed herein is a system and method to automatically identifysub-areas of high potential interest to the user in images of the entirearea under observation and to automatically determine and executeflight- and/or ground-paths of an Unmanned Aerial or Ground vehicle toobtain high-resolution pictures of those areas. Furthermore, the systemlearns from users' feedback and improves its ability to acquire andpresent high-value images to the users.

The cyber-physical system includes a set of algorithms and theirsoftware implementations operating on physical hardware in a real-worldunstructured environment. The system can determine, such as inreal-time, the most interesting locations to examine in detail byanalyzing visual data from high-altitude images using resources on-boardand/or on a connected computer. The system can then develop a path forthe vehicle to obtain close-up images of these potentialhigh-information locations. The interesting locations are found using aset of image processing algorithms including, but not limited to,spectral analysis, neural network analysis, and/or nonparametricclustering algorithms. Reinforcement learning algorithms that aretrained using user inputs can continually improve the utility of thesystem. Although the present disclosure describes examples ofapplications in agricultural applications, it can be adapted/extendedand applied to non-agriculture applications including ranching,forestry, natural environments, and infrastructure monitoring. Inaddition to monitoring crops for any potential problems and quality ofgrowth, it can also be used for stand counts and weed identification,for example.

FIG. 1 shows the general flow of an example of a cyber-physical systemthat can monitor unstructured environments. The individual elements andassociated algorithms of the cyber-physical system are now described inone embodiment.

At 101, the user can indicate the area of interest and a flight plan canbe planned automatically. High-altitude images can be acquired using agrid based flight pattern. In one embodiment, this process can take intoaccount the field-of-view and/or lens-distortion of the camera and theaircraft altitude to take the least number of geotagged images thatcompletely cover the entire area of interest.

At 102, image processing can be performed. As an example, geolocationsassociated with each of an arbitrary number of sub-areas of eachhigh-altitude image can be identified. Images can also be processed tocorrect for lens-distortion (which may not be required by the algorithmsdescribed in the following), and/or color correction. Image analysis canbe performed. For example, one or more automatic segmentationalgorithms, including but not limited to a newly developedgrid-comparison algorithm and a Dirichlet process K-means algorithm, canbe employed to identify different features associated with differentparts of images. Transformations may be applied to the images to enablespecific feature detection algorithms and improve computationalefficiency, including, but not limited to RGB-to-HSV transformation. Thefeatures analyzed for detecting high-information areas can includespectral information, spatial gradients in spectral characteristics,object or shape recognition, and image structure. Planning flight-pathcan be performed to obtain close-up images. As an example, theinteresting images, such as tagged by users, are associated withgeographical locations. An algorithm for optimally planning a route thatvisits all of those locations is autonomously planned.

At 103 and 104, the UAV and/or ground vehicle executes that path usingGPS, INS, or other sensing aids to obtain the target close-up images ofpotentially high value locations, and user feedback can be obtained.Users can be shown images that they are most likely to find important,potentially along with images with undetermined predicted utility. Userfeedback or rating of images, such as thumbs up or thumbs-down rating,or other scales of rating, can be obtained. Users can also provide alabel in the form of a voice note, a written label, or by preselectingfrom an existing set of labels.

At 105, the system can be adapted to find the most interesting images.For example, a learning element including but not limited to NeuralNetwork, Gaussian Process, Support Vector Machine, or nonparametricclustering algorithm, can be used to automatically identify whichimage-feature clusters users are most likely to find interesting.

One or more benefits of the present disclosure are, for example,autonomous UAS mission planning will enable agronomists, ranchers, parkrangers, etc. to use UAS with minimal technical expertise. Immediatedelivery of specific actionable information will improve the speed andeffectiveness of response to the particular issues. The UAS based systemwill enable complete monitoring of each field or area, rather than therandom-sampling approach taken during ‘on-foot’ scouting. Field-scoutingtime will be reduced, such as from about an hour per field for on-footscouting to about 15 minutes. The use of consumer UASs andvisual-spectrum cameras will substantially reduce the cost of aerialmonitoring Tablet or cell phone application.

In one embodiment, a tablet application (app) (or application executedby another mobile device) can present to the agronomist a number of(e.g., about 10) close-up photographs of potential problem areas in afield obtained during a short (e.g., about 15 minutes) and fullyautonomous mission of a consumer drone (e.g., equipped only with avisual spectrum camera). The agronomists can use their domain knowledgeand experience to interpret these images and decide on the best courseof action. Although a mobile device is described, the application can beexecuted on other computing devices including a desktop computer.

In contrast with existing approaches, this approach can make it possibleto reduce the total field-monitoring time for a typical 80 acre field tounder 15 minutes from UAS launch to presentation of high-value images.In comparison, a typical human agronomist team can scout a field in 1 to2 hours. While existing drone-based approaches require about half anhour to capture images, the requirement of uploading the data to cloudfor analysis adds several hours to days for the total workflow, delayingrevelation of potential issues.

Conventional, or “on-foot”, crop scouting is often carried out bytrained teams with members that are experts in various aspects ofagronomic issues such as weeds, fungal diseases, insect pests, etc. Dueto the need for monitoring a large number of fields during criticaltimes in the growing season agronomists have to rely on sampling smallareas of the fields, rather than thoroughly surveying each field. Whilethis approach is successful in routine inspections and spotting mostproblems, it can miss areas of concern. Moreover, when the soybeancanopy begins to close, on-foot inspection is not feasible, though majorproblems can still arise.

With respect to aerial field-monitoring, farmers and agronomists haveconsistently asserted throughout customer-discovery interactions that auseful crop monitoring product does not yet exist, despite numerouscommercial offerings. It is believed that the recent rapid drop in thecost of UASs has led to a massive increase in the commercial activity,jumping ahead of the actual capabilities of the technology that arenecessary for critical product features. Commercial offerings have alsoimplemented post-processing like stitching [1] and orthorectification[2, 3] that are traditional satellite based remote sensing, but havelittle agronomic relevance. Conversely, the vast majority of academicresearch on aerial monitoring has focused on using data fromdevelopmental sensors (hyper- or multi-spectral cameras) that are notyet ready for reliable use.

However, agronomists who have inspected fields with simple cameras ondrones have found that they can reliably spot problem areas from thevisual images and examine them by manually controlling flights. Ofcourse, this approach is quite labor intensive and therefore not costeffective. Automating and improving drone-based visual scouting istherefore a compelling approach to improving the effectiveness andproductivity of agronomists.

With respect to academic research on aerial field-monitoring: There hasbeen a reliance on expensive sensors in academic research on aerialmonitoring and remote sensing of agriculture. The bulk of the researchhas been based on data from multi-spectral [3, 4] and hyperspectral [5,6, 7] imagery. Laser Interferometric Detection and Ranging (LIDAR) [8,9], fluorescence spectroscopy [6, 10], and thermography [11, 12] havealso been investigated as potential agriculturally useful sensors. Thisreliance on advanced sensors may be an artifact of the historical workin satellite based remote sensing, which involved techniques likeNormalized Difference Vegetation Index (NDVI) for continental-scaleenvironmental health assessments [13, 14]. However, the major hurdles tothe use of such sensors is their cost, complexity, and weight. Costlysensors (often over $10,000), that require a high degree of technicalexpertise to operate and interpret the data from are impractical for usein conventional commodity crop agriculture.

UAS borne visual-spectrum sensors have also been used successfully tocollect useful data from agriculture, including for predicting yields[15], phenotyping [16], evaluating the health of the plants [17], andidentifying weeds [18]. These results, as well as the experience ofagronomists in successfully spotting problem areas in the field visuallyfrom drone survey forms the foundation for our proposed approach.

With respect to commercial offerings for agricultural monitoring: Whilethere is currently increasing amount of commercial activity inagricultural drones, a cohesive and useful solution has not emerged. Abrief description of major participants in UAS based agriculturalmonitoring follows: Drone hardware companies (e.g. DJI, 3DR) makeconsumer-friendly drones, but it is believed that the do not providemission automation, sensing, or actionable insights. Flight managementservices, (e.g. DroneDeploy) offers mission planning software thatgenerates basic flight plans, but it is believed that these flight planscannot deliver actionable information to the farmers. Sensor companies(e.g. Sentera, MicaSense) are offering multi-spectral sensing which itis believed that most farmers find neither usable nor actionable.Satellite data (e.g. Planet Labs) is too low-resolution for farmers toreliably gauge even in-season trends, let alone deliver specificallyactionable information. Finally, companies that integrate severalelements (e.g. PrecisionHawk) or that provide cloud-based data analysis(e.g. AgLytix) need large data sets to be uploaded to their servers,which is highly impractical in agricultural locations, the majority ofwhich have very low bandwidth connections to the Internet, if they arenot completely off-grid. The efforts on data analysis part of this valuechain have perhaps been the most misdirected. Techniques like stitching(also known as mosaicking) [1] and orthorectification [2], and NDVI[13], in particular are pursued as a standard part of the agriculturalmonitoring toolkit due to their historical tradition of use insatellite-based remote sensing, even though their effectiveness inimproving agronomic outcomes is at best questionable.

Adaptive Anomaly Detection of Agricultural Fields

This section describes algorithms that can be used in an adaptiveanomaly detection system of the present disclosure for agriculturalfields. The system aims at efficiently detecting anomalous regions on alarge crop farm using off-the-shelf cameras carried by unmanned aerialvehicles (UAV). The UAV first performs a uniform grid scan over the farmfield at a high altitude. The images collected at constant intervals arethen processed to determine key regions of interest (ROI) that arepotentially anomalous. Next, the path planning algorithm uses thepositions of ROI to generate the route for low-altitude flight.

Distortion Removal

In order to maximize the area covered in each image, the camera on thedrone can be equipped with wide angle lenses which create significantdistortion around the periphery. Thus, the first step is to remove theartifact caused by lens distortion. The camera is calibrated before eachflight, such as using an open source calibration routine based on thework by Zhang (see Zhang, Computer Vision, 1999. The Proceedings of theSeventh IEEE International Conference on, 666-673. IEEE. 1999), althoughpother calibration techniques can be utilized. The calibration algorithmfinds the optimal camera parameters and distortion coefficients thatminimize the reprojection error between observed image points and theirknown 3D world coordinates. In the calibration procedure, a chessboardpattern is recorded in at least 20 orientations. The corners aredetected and used to determine the optimal camera parameters based onall 20 images. The reprojection residual from the optimal cameraparameters is on the order of 0.1 pixel.

Once the camera parameters and distortion coefficients are determined,each pixel (u,v) in the corrected image is mapped to the correspondingcoordinate in the original image by a non-liner function of theparameters to remove the distortion. FIGS. 7A and 7B illustratedistortion removal performed by the exemplary method. In FIG. 7A, theoriginal raw image has significant distortion around the periphery. InFIG. 7B, the undistorted image is generated where the rows becomeproperly straightened after correction.

Classical clustering algorithms, such as k-means, requires appropriatechoice of the number of clusters which is often unknown a priori. TheDP-means algorithm utilized in one or more embodiments, eliminates suchrequirement by including a penalty for the number of clusters.

Clustering

The anomaly detection algorithm is based on the assumption that farmfields are generally homogeneous with few scattered deviations. Aclustering algorithm can effectively separate these anomalies from themajority of the image. A Bayesian non-parametric clustering algorithmnamed DP-means is applied. First an image is converted to HSV (hue,saturation, and value) color space. We then apply a Gaussian blur tosmooth out small scale spatial variations. Finally, the pixels areclustered according to their HSV values. The algorithm iterativelyreduces the objective function until local convergence:

${\sum\limits_{c = 1}^{k}\;{\sum\limits_{x_{\ell} \in \ell_{c}}{{x_{i} - \mu_{c}}}^{2}}} + {\lambda\; k}$

Algorithm 1: DP-means Input: x₁, ..., x_(n) → data Parameter: λ →cluster penalty Output:

₁, ...,

_(k) → clusters μ₁, ..., μ_(k) → clusters centroids z₁, ..., z_(k) →clusters labels Initialize: k = 1

₁ = {x₁, ..., x_(n)} $\mu_{1} = {\frac{1}{n}{\sum_{i = 1}^{n}x_{i}}}$z_(i) = 1∀i = 1, ..., n while not converged do  foreach x_(i) do  Compute d_(ic) = ||x_(i) − μ_(c)||² for c = 1, ..., k   ${{if}\mspace{14mu}{\min\limits_{o}d_{ic}}} > {\lambda\mspace{14mu}{then}}$   k = k + 1    z_(i) = k    μ_(k) = x_(i)   end   else    z_(i) =argmin_(c)d_(ic)   end  end  Update clusters:

_(j) = {x_(i)|z_(i) = j}  foreach

_(i) do   ${{Update}\mspace{14mu}{centroids}\text{:}\mspace{14mu}\mu_{j}} = {\frac{1}{\ell_{j}}\sum_{x_{i} \in {\ell_{j}x_{i}}}}$ end end

The result of the DP-means are shown in FIGS. 8A-8B.

ROI Identification

Adjacent pixels belonging to the same cluster are agglomerated intoblobs of various sizes. Each blob is a potential ROI. The can be rankedby a function of size and deviation from the global mean to identify themost significant feature. Top N ROIs are output to the path planningalgorithm for the subsequent low-altitude flight. FIG. 9 shows the top 5ROIs from the clustered image.

Path Algorithm

This section describes the functions and path-planning algorithmutilized in an adaptive anomaly detection system of agricultural fields.The path-planning algorithm uses the positions of the key regions ofinterest (ROI) to generate a route for low-altitude flight.

Pre-processing data

Converting Pixel Coordinates to Geodesic Coordinates

To determine the path between ROIs in different images, the pixelcoordinates of ROIs can be converted into geodesic coordinates. This isdone by using the altitude logged in the high-altitude flight, the pixelsize of the image, the geodesic coordinates of the center of the image,and the field of view (FOV) of the camera in the following equations:

$\begin{matrix}{\mspace{76mu}{{{Horizontal}\mspace{14mu}{Pixel}\mspace{14mu}{Distance}} = {2*{Altitude}\frac{\tan\left( \frac{horizontalFOV}{2} \right)}{ImageWidth}}}} & (1) \\{\mspace{76mu}{{{Vertical}\mspace{14mu}{Pixel}\mspace{14mu}{Distance}} = {2*{Altitude}\frac{\tan\left( \frac{verticalFOV}{2} \right)}{ImageHeight}}}} & (2) \\{\mspace{76mu}{{Lat}_{2} = {\arcsin\left( {{{\sin\left( {Lat}_{1} \right)}{\cos\left( A_{D} \right)}} + {{\cos\left( {Lat}_{1} \right)}{\sin\left( A_{D} \right)}{\cos(\theta)}}} \right.}}} & (3) \\{{Lon}_{2} = {{Lon}_{1} + {\arctan\; 2\left( {{{\sin(\theta)}{\sin\left( A_{D} \right)}{\cos\left( {{Lat}\; 1} \right)}},{{\cos\left( A_{D} \right)} - {{\sin\left( {Lat}_{1} \right)}{\sin\left( {Lat}_{2} \right)}}}} \right)}}} & (4)\end{matrix}$

Equations 1 and 2 calculate the horizontal and vertical (or width andheight respectively) of individual pixels. Equations 3 and 4 determinethe geodesic coordinates (Latitude, Longitude) of individual pixelsbased off of the latitude and longitude of the center of the image.

Altitude is the altitude in meters that the high-altitude flight imagewas obtained at. FOV stands for ‘Field of View’ of the camera or theangular field of view of the sensor. Lat₂, Lon₂ are the latitude andlongitude of the individual pixel. Lat₁, Lon₁ are the latitude andlongitude of the center of the image. A_(D) is the angular distancebetween two geodesic points which is calculated by dividing the radialdistance between the two points (distance you could measure on a map) bythe radius of the earth. θ is the bearing between the two pointsdetermined by using arctan 2(X,Y) where X is the horizontal distancebetween two points in an image and Y is the vertical distance betweentwo points in an image.

Ranking ROIs

Currently, ROIs are ranked by their area to determine the number of topN ROIs. Other features of images such as location, differentiation ofindividual anomaly types, etc. can be used to rank ROIs, alone or incombination with other features.

Developing the Path Planning in the Adaptive Path-Planning Algorithm

The geodesic coordinates and altitude (needed to obtain an image of theentire ROI) are inputs for the Path-Planning Algorithm. The algorithmconsists of two parts: a nearest-neighbor search to initialize asub-optimal path and a genetic solver for the Traveling Salesman Problem(TSP). Initially, only the genetic solver for the TSP was utilized tocreate a path for the low-flight. Later, the Nearest Neighbor SearchAlgorithm was considered and then initialized as a pre-processing stepfor the Genetic Path Planning Algorithm.

Nearest Neighbor Search Algorithm

The Nearest Neighbor Search Algorithm is a greedy algorithm that findslocal optimum solutions which does not always yield a global optimumsolution.

The geodesic coordinates and inspection altitude are inputs for thisalgorithm which outputs the waypoints for the Inspection Flight.

X₀ is initialized automatically as the “Start” node from where the UAVends the high-altitude Scout Flight.

X_(N+1) is the “Home” node that is previously obtained from the launchpoint of the Scout Flight.

dmat is the Distance Matrix for all nodes X and is initialized withdistance calculations between the geodesic coordinates of each node toall other nodes including the “Start” and “Home” node. It is a [N×2,N×2] matrix that includes the value of the edges between “Start” and“Home” node.

R is the path solution to visit the N ROI nodes in the low-altitudeInspection Flight.

The distance matrix dmat is initialized along with the set of nodes tovisit M excluding the Start node and Home node. To optimize the NearestNeighbor Search solution, the search is performed both “Forwards” and“Backwards”. By this, “Forwards” is from Start to Home node and“Backwards” is the reverse of these endpoints. This is performed becausethe “Forward” search doesn't account for the distance from the nodes inM to the Home node. Thus, in certain scenarios the “Forward” route maynot always yield a shorter solution than the “Backwards” route.

The “first” iteration is done outside the while loop by finding theshortest distance between the initial node and the first closest node nin M. This node n is then removed from set M and added to final path Rbefore being designated as previous node p.

In the “while” loop, the same process in the “first” iteration iscarried out until the set of nodes to visit M is empty. Then the “end”node (which could be either the Start node or the Home node), is addedto the route and the distance between the last node in M and the “end”node added is tabulated. Finally, the shorter route of the “Forward” (S)and “Backwards” (R) route is sent as output for the final path solutionfor the Inspection Flight.

Algorithm 1: Nearest Neighbor Search Algorithm Input  : X₁,...,X_(N) →geodesic coordinates (Lat,Lon) of the centers of N ROI Nodes   :A₁,...,A_(N) → Inspection altitude of top N ROIs. Output   : InspectionFlight Path Waypoints (Coordinates & Altitude) Initialize : X₀ = StartNode Coordinates (from where UAV ends Scout Flight) X_(N+1) = Home  NodeCoordinates (UAV ends Inspection Flight here) dmat=MakeDistMat(X₀,...,X_(N+1)) → Make distance matrix between nodes Set dmat(r , c) = ∞ , for all (r,c) where dmat(r , c) = 0  Setdmat(1,N+2) = ∞ and dmat(N+2,1)=∞→ Edges between X₀ and X_(N+1) forf=1:2 do | if f==1 | then | | first=N | | else | | | first | =|N+2 | |end | end | M={2,..., N+1}→ Nodes to visit | (distmin,n) = dmat(n,first)→ Next node n to visit | M(n)=[ ] → Removes Node n from Search |R=[first,M(n)] → Add node in order to Forward | Route | p = n → NextNode n is now Previous Node p | while M is not empty do | | (distmin,n)= dmat(n,p) → Next node n to visit | | M(n)=[ ] → Removes Node fromSearch | | R=[R,M(n)] → Add node in order to | | Route p = n | end |  R= [R,first] → Add last node in | Route | if f==1 then | | S=R | | else || | R=Reverse(R) → Reverse route so it begins at Start node and ends atHome node | end end end if Reverse Route (R) is shortest route t|hen$\left| \mspace{11mu}{InspectionFlight} \right. = {{\begin{bmatrix}X_{R_{1}} & X_{A_{1}} \\\vdots & \vdots \\X_{R} & X_{A}\end{bmatrix}\mspace{14mu}{else}\mspace{14mu}{InspectionFlight}} = \begin{bmatrix}X_{s_{1}} & X_{s_{1}} \\\vdots & \vdots \\X_{s} & X_{s}\end{bmatrix}}$

Genetic Algorithm to solve Fixed Endpoints-Traveling Salesman Problem(TSP)

The genetic solver for TSP determines the shortest path by process ofmutation and elimination of randomized “Genes.” The “Genes” in thissituation are the different variations of possible paths through Nregions of interest (ROIs). Demonstrated in FIG. 10, there are “Parents”who each have offspring A, B, C, and D. Each offspring has at least oneunique difference. The order of these offspring are randomized beforetesting. 4 offspring are selected at a time for strength testing. Theseoffspring or TestGenes (different variations of the path) are tested forminimum distance. The strongest offspring (or the TestGene that yieldsthe shortest path) is kept while the other 3 are discarded. The “Genes”of this strongest offspring are mutated to yield another 4 offspring inthe “new” generation.

The “Genes” of the “old” generation are “mutated” via randomly selectingtwo nodes, excluding the first and last node, in the path and executingone of the three “mutations”:

Null: No mutation is made.

Flip: Reversing the order of the nodes between the two ‘mutation’ nodes,including the two ‘mutation’ nodes.

Swap: Exchange the position of the two selected nodes.

Slide: Move the first ‘mutation’ node (‘first’ as in the first selectednode relative to the ‘second’ node in the sequential order) to the endof the order of nodes between the two ‘mutation’ nodes while moving theother nodes as well. This means that nodes in the selected range ofnodes (nodes between the two ‘mutation’ nodes) move to a lower or‘earlier’ position in the order OriginalPosition−1. As a result, in theselected range of nodes: the first node becomes the last node; thesecond node becomes the first node; the third node becomes the secondnode, and all the way up to the last node now becoming the second tolast node.

Observe the example below:

OriginalTestPath=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Mutation Node 1=OriginalTestPath(2)

Mutation Node 2=OriginalTestPath(6)

MutatedTestPath=Slide(OriginalTestPath)

MutatedTestPath=[1, 3, 4, 5, 6, 2, 7, 8, 9, 10]

One “Gene” is left unchanged; thus, three of the offspring will bemutated variations of the parent and one offspring will be an exact copyof the parent. The other offspring in the current generation undergo thesame process via selecting 4 offspring at a time until the entirecurrent generation is processed. The new generation then becomes thecurrent generation and is then randomized for selection beforeundergoing the above process again. This process is repeated for Iiterations. The best solution obtained at the end of these I iterationsis used as the final path solution.

Algorithm 2: Genetic Solver for the Traveling Salesman Problem Input:X_(1, . . . ,) X_(N)→ coordinates (Lat,Lon) of the centers of the Top NROI Nodes  A_(1, . . . ,) A_(N)→Inspection altitude of Top N ROIs.Output:   Inspection Flight Path Waypoints (Coordinates & Altitude)Initialize: X₀ = Start Node Coordinates (from where UAV endshigh-altitude flight) X_(N+1) = Home Node Coordinates (UAV endslow-altitude flight here) dmat=MakeDistMat(X_(0, . . . ,)X_(N+1)) → Makedistance matrix between nodes Set dmat(r , c) = ∞, for all (r,c) wheredmat(r , c) = 0 Set dmat(1,N+2)=∞ and dmat(N+2,1)=∞→ Edges between X₀and X_(N+1) TestPaths=[4 × N Array of Zeros] TestGenes=[4 × N Array ofZeros] G₁=[100 × N Array of Zeros] → Previous Generation G₂=[100 × NArray of Zeros] → Next Generation distTestPaths=[0,0,0,0] → Placeholderfor distance of TestPaths CurrentBest=X_(1, . . . ,)X_(N) → Initializeroute solution for i =1 : I do | G₁=randPerm(CurrentBest) → GenesAssembled with Random Permutations of Path | randSelection =randPerm(100) → Randomize order of selection of Genes | for j = 4 : 4 :100 do | | TestGenes = G₁(randSelection((j − 3) : j)) → Select 4 genesat a time | | for d = 1 : 4 do| |  | distTestPaths(d)=dmat(1,TestGenes(d,1)) | |  | for n = (2 : (N))do| |  | | distTestPaths(d)=distTestPaths(d)+dmanTestGenes(d,n−1),TestGenes(d,n))end | |  | distTestPaths(d)=dmat(TestGenes(d,n)),N+1) | | end| | (distmin,ID)=min(distTestPaths) → Find shortest route| | MutationNodes=randint(2,[2:N]) → Randomly select two integersbetween 2 and N | | CurrentBest=TestGenes(ID, :) | | TestPaths(1,:)=CurrentBest | | TestPaths(2, :)=Flip(CurrentBest,MutationNodes)| | TestPaths(3, :)=Swap(CurrentBest,MutationNodes) | | TestPaths(4,:)=Slide(CurrentBest,MutationNodes) | | G₂((j−3):j, | |:)=TestPaths| |end | |G₁((j−3):j, :)=G₂((j−3):j, :) → Replace ”Genes” in G₁ withoffspring's ”Genes” from G₂ |end |R = [X₀,CurrentBest, X_(N+1)] |${InspectionFlight} = \begin{bmatrix}X_{R_{2}} & A_{R_{1}} \\\vdots & \vdots \\X_{R} & A_{R}\end{bmatrix}$

First Comparison Analysis

A comparison analysis between the computation times, distance, andtravel times of the nearest neighbor search and the genetic solver wasperformed with 1000 reps for 10, 20, and 50 points randomly generatedalong a uniform distribution.

Genetic Solver Parameter Analysis

To improve the genetic solver, the solver's gene population sizeparameter and number of points input parameter were evaluated. 10, 20,and 50 uniformly distributed points were used with population sizes of100, 200, and 500 to gauge the effect both had together on the amount ofiterations needed for convergence, computation time, and travel time. 40reps were performed for each of the sixteen levels of the test. Thedefault number of iterations used for all reps was 5000 iterations.

Based off the results of the genetic solver's parameter test, apre-processing step was added to the genetic solver by using thenearest-neighbor algorithm to generate an initial sub-optimal path toinput. This was predicted to minimize the amount of iterations neededoverall in the algorithm and reduce computation time. Another additionto the input pre-processing was minimizing the amount of iterationsneeded depending on the number of points inputted. For example, if thereare 10 points we may only need 500 iterations, but if there are +50points we may need 5000 iterations.

Second Comparison Analysis

A second comparison test between the computation times and travel timesof the nearest-neighbor search and the altered genetic solver wasperformed. It involved executing 3000 reps for sets of 10, 20, and 50points randomly generated across Uniform and Gaussian distributions. Forexample, the addition of testing points generated along a Gaussiandistribution can be indicative of whether the distribution of the pointshad an affect on the performances of the algorithms.

EXAMPLES Example 1. Feasibility

The preliminary results demonstrate feasibility. Each algorithm has beenevaluated with large real-world datasets obtained using representativehardware (3DR Solo UAS and GoPro Hero4 camera) on Soybean farms aroundIllinois. The algorithms described herein were thoroughly evaluated andtheir performance benchmarked. For example, FIG. 3 depictsimplementation of a Deviance Detection Algorithm (DDA) which masks allnon-anomalous areas of an image. The two interesting areas detected inthe image (a small one near the left edge and one right of center) wereidentified by DDA in just under a second on a Macbook pro computer.Furthermore DDA can easily distinguish agricultural and non-agriculturalfeatures, such as the road towards bottom right of the image.

Example 2. In-Situ Image Analysis

Reducing computational complexity for in-situ image analysis can beachieved primarily by utilizing easy to compute pixel specific features,such as, the pixel Hue-Saturation-Value (HSV) parameters [22] averagedover blocks of the image. This simple approach avoids the computationalburden inherent in more advanced machine vision algorithms such asobject recognition, feature extraction, edge detection. Using largereal-world Soybean farm datasets, the exemplary process shows thatanomalous areas can be quickly and reliably identified in acomputationally efficient manner by simply identifying clusters ofpixels that have different HSV values than the average (FIGS. 3, 4).Unlike generic object recognition or feature extraction machine visionalgorithms often used in agricultural monitoring [23, 24], the exemplaryembodiments utilize largely homogeneous spectral structure of modernrow-crop agriculture. Leveraging this structure, the exemplaryembodiments focus on identifying minute variations in HSV variables ofgroups of pixels that are difficult to spot with the naked eye, asdescribed herein. Identifying variations in visual spectrum can be themost practical method for autonomous aerial agricultural monitoringuntil multispectral and hyperspectral cameras become substantiallycheaper and lighter [25, 26, 27].

Reducing effective image resolution pre-analysis can be another approachfor improving computational feasibility of nonparametric clusteringalgorithms that can improve speed without reducing detection accuracy.The exemplary embodiment of the Deviance Detection Algorithm alreadyutilizes gridding, analyzing the image in a 100×100 matrix, which takesonly 0.78 seconds even for a non-parallelized software implementation ona laptop computer (FIGS. 3 and 4). The speed of DP-means clustering [21](FIG. 4 (middle image) improves non-linearly with decreasing resolutionwithout any appreciable change in cluster detection $20% and $100%(FIGS. 5A-C). These results indicate that significant computationalefficiency can be achieved by using low resolution images.

Example 3. Adaptive Two-Stage Flight Planning Algorithms for AutonomousMonitoring

The path planning algorithms of the exemplary embodiments can adapt thewaypoints in the second stage of the mission to prioritize obtaininghigh-resolution images of the most interesting parts of the farm usingthe images from the first stage of the mission, while stayingconservatively within the remaining flight endurance.

In the first stage of the mission, the aircraft can perform an automatedgrid-pattern based scan from about 100 feet above ground level andrecords images with a 10% area overlap. The images can be transmitted tothe app on a mobile device using functionality already available onleading consumer UAS, including, but not limited to, 3DR Solo and DJIPhantom. Using algorithms developed above, potential interesting areasare identified from the transmitted images. The software thenautomatically plans a path in the second stage of the mission to takehigh-resolution close-up images of the interesting areas from about 30to 50 feet. FIG. 6 depicts a generated flight-path solution on anexample dataset obtained at a Soybean farm near Ogden, Ill. The datasetconsisted of images recorded during the stage-1 grid path. The imageanalysis and stage-2 path planning was done using the exemplaryalgorithms, but post-flight. The image analysis and path planningalgorithms can be within a tablet app for complete on-site execution. InFIG. 6, the white trace shows the stage-1 grid path. A total of 20interesting areas were found, their locations and associated figures areshown on the path. The red trace shows the stage-2 path that was createdby our preliminary path planning algorithm where the UAS visits 10 ofthe interesting areas to obtain closeup images.

The path-planning algorithms can be further designed to optimize themission plan such that a maximum number of interesting images areobtained within the flight endurance of the UAS. The waypoints can bedesigned such that the UAS can start from the end point of the highaltitude grid survey, reduce altitude for acquiring close-up pictures,and return to the original launch point at the end of this second stageof the mission. Such a path planning problem can be cast as that ofsearching for the least-cost Hamiltonian path that visits apre-specified number of the identified interesting locations. Solutionsto Hamiltonian path planning problems are NP-complete [34, 35], hence,heuristics such as distribution of waypoints in different parts of thefield can be utilized to further speed up the solution times. Suchapproximations have been known to significantly improve thecomputational speeds for UAS applications [36].

Additionally, the algorithm development can implement the ability to usealtitude between 30 to 50 feet as a control variable to attempt tocapture each interesting location in one image where possible. Thisapproach is utilized to minimize cost and weight of incorporating acamera with an optical zoom, however, when a camera with real-timecontrol of optical zoom is available under at comparable prices to theGoPro hero (around $400), the software can be easily modified.

Example 4. Tablet Application and User-Interface Development

The application can have an interface where the grower has minimumparameter selection or tuning to do which will improve productivity.Accordingly, the workflow can have three main stages. In a one-time“SETUP”, the agronomist can draw the boundaries of a field using adrag-and-draw interface overlaid on mapping systems, such as Google mapsusing available APIs. The second stage of the workflow, “SURVEY,” can beexecution of the monitoring flight. In this stage, the agronomistpresses the FLY button on the app after having setup the UAS. The appcan automatically plan the flight path based on the field boundaries,execute the first stage of the two-stage adaptive mission describedabove, then transmit the imagery back to the app for processing, and getan updated mission plan for the second stage of the mission. The thirdstage of the workflow, “INSPECTION,” can present the grower with, forexample, about 10 close-up images. Here, the agronomist will provide athumbs up or down feedback on each close-up image before moving to thenext image. In addition, the agronomist can also record text or voicenotes about each image. The agronomist will have the ability to view andtag any of the images taken by the UAS after this stage. All images fromthe mission can be overlaid on the map in the appropriate location.Based on user feedback, the application can be altered to make it moreuser-friendly and aesthetic.

Example 5. First Comparative Test

Observing FIG. 11, the genetic solver yielded shorter routes than thenearest neighbor search with average computation times of 3.2236,3.2245, and 3.8332 seconds on average for 10, 20, and 50 points withsome outliers greater than 6 seconds (see FIG. 13). However, thenearest-neighbor algorithm's solution has an average computation time inthe hundredths and millionths of seconds (see FIG. 14) for 10, 20, and50 points. This resulted in instances where the total travel time alongthe path from the genetic solver's solution was equal to or longer (seeFIG. 12) than that of nearest neighbor search's solution. At only 10points, the genetic solver's solution yielded a shorter travel time 57%of the time. As the number of points increased, the genetic solverbecomes substantially superior to the nearest neighbor search with thetravel time of the genetic solver's solution being shorter than thenearest neighbor solution 92.8% and 96.8% of the time for 20 and 50points respectively. FIG. 13 illustrates Computation time in seconds ofthe Genetic Solver for the Traveling Salesman Problem (TSP) for 10, 20,and 50 Uniformly Distributed Points.

Genetic Solver Parameter Analysis

The genetic solver was used to solve the Traveling Salesman Problem for10, 20, and 50 points with population sizes of 100, 200, and 500. Asseen in FIG. 16 the amount of iterations necessary for the solver toconverge on a solution increases with the number of points. However, asthe population size increases, the amount of minimum iterations neededdecreases. This decrease is noticeable, but not significant enough toaffect the total computation time and resulting travel time. FIG. 15displays the computation time (if minimum iterations are used) for thecombinations of Population Sizes and Number of Points. For each numberof points, the computation time is almost always the least at apopulation size of 100.

The largest concern is the effect of population size and iterations ontravel time. The percent quality difference in travel time between eachrespective set of points with different population sizes is describedherein. This was determined by comparing the respective travel time(Distance/5 assuming the UAV travels at 5 m/s) of each rep of pointswith different population sizes for the same set of points.

$\frac{{T\left( V_{NP} \right)} - {\min\left( {T\left( V_{NP} \right)} \right.}}{\min\left( {T\left( V_{NP} \right)} \right)}$

This is calculated as:

where T is the Travel Time for a given combination V of N number ofpoints and population size of P.

FIG. 17 shows that every instance where the population of 100 was used,its respective travel time was the least compared to that of the samepoints with population sizes of 200 and 500. Looking at the hypotheticalpercent quality difference where the minimum amount of iterations isused in each instance (see FIG. 18) this is still true for a majority ofthe repetitions.

From this it seems that it would be most optimal to use a genepopulation size of 100. For reducing computation time via controllingthe number of iterations, the following was added to the Genetic Solverfor:

For N<10 points, I (iterations)=500

For 10≥N<15, I=1000

For 15≥N<30, I=3000

For N≥30, I=5000

This was done based off the results obtained in FIG. 16, but stillarbitrarily assigned to also account for potential outliers.

Below is shown Table 1: Comparison of travel time and distance inOriginal Genetic Path Planning Algorithm and Nearest Neighbor SearchAlgorithm for sets of 10, 20, and 50 points.

Travel Time PQD Distance PQD NN Comp Time GA Comp Time Pts 10  20 50 1020 50 10 20 50 10 20 50 Max   57.30%   66.41%   57.32%   62.31%   70.55%  61.53% 2.87E−2 1.63E−3 6.27E−3 18.54  4.79 8.51 UOB   17.50%   36.56%  49.06%   20.94%   39.32%   52.24% 8.43E−4 8.83E−4 2.00E−3 3.79 3.714.35 Q3    5.53%   17.23%   25.78%    8.51%   10.72%   28.12% 6.09E−47.92E−4 1.70E−3 3.23 3.33 3.80 Q2    6.91%    9.55%   17.62%    3.91%  12.03%   19.68% 4.77E−4 7.61E−4 1.71E−3 2.97 3.21 3.73 Q1  −2.46%   4.34%   10.21%    0.23%    6.66%   12.04% 4.52E−4 7.31E−4 1.64E−32.86 3.08 3.59 LOB −14.43% −15.00% −13.10% −12.20% −12.94% −12.08%2.18E−4 6.39E−4 1.43E−3 2.31 2.70 3.15 MIN −11.01%   10.27%   11.71% −5.60%   10.49% −11.95% 4.00E−4 8.97E−5 3.16E−4 1.85 0.21 0.49 AVG   3.09%   11.74%   18.64%    6.22%   14.24% −20.76% 6.05E−4 7.73E−41.78E−3 3.22 3.22 3.83 St. Dev    8.06%   10.27%   11.71%    8.15%  10.49% −11.95% 1.01E−3 8.97E−5 3.16E−4 0.87 0.21 0.49

Distance and travel times were compared by using percent qualitydifference between the respective values of the Nearest Neighbor Searchand the Genetic Path Solver. Percent quality difference (PDQ) iscalculated as (xobserved−xreference)/xreference where xreference is thevalue of the variable for the Genetic Path Solver in this situation.

Second Comparison Analysis

With the modified genetic solver, another comparison analysis wasperformed to compare the genetic solver and the nearest neighborsolution. Comparing the computation time of the genetic solver from thefirst and second comparison analysis in FIG. 20 and FIG. 13, thedistributions remain roughly the same from the first to the secondanalysis. The percent quality difference in travel times differs in thesecond test compared to the previous test. FIG. 19 illustratesComputation time for the Nearest Neighbor Search Algorithm for pointsdistributed along Uniform and Gaussian distributions in the SecondComparison Analysis.

In one or more embodiments, the current iteration of the path-planningalgorithm can run a nearest-neighbor search as an initial guess of asub-optimal path for the genetic solver. If the genetic solvercomputation time lasts longer than the predicted benefit in travel timefrom its solution, then the nearest-neighbor algorithm can be executedas the flight path if it yields a more optimal travel time than thecurrent solution the genetic solver has determined.

In one or more embodiments, the genetic solver algorithm can be stoppedafter it has improved past the Nearest Neighbor Search's solution by adefined percentage. In another embodiment, tracking can be performed(and utilized for controlling execution of the algorithm) for how manyiterations it takes for each improvement in the path length to occur.

In FIG. 22, one embodiment of a workflow showing a user interface 2200of an exemplary embodiment for monitoring of unstructured environments.The workflow enables selection of a search area, automated flight, andstatus updates regarding images captured and other parameters of theUAV. The workflow further enables the second automated flight, andstatus updates regarding target images captured and other parameters ofthe UAV.

FIG. 23 illustrates a workflow showing ability of a user to providefeedback via a user interface 2300 of an exemplary embodiment formonitoring of unstructured environments. The workflow enables scrollingthrough captured images, information regarding captured image location,selection or ranking or approval (e.g. a thumbs-up icon) of the images,and inputting specific comments regarding each image.

FIG. 24 illustrates a comparison between a commercial service workflow2430 and a workflow 2460 showing ability of a user to provide feedbackof an exemplary embodiment for monitoring of unstructured environments.As can be seen from FIG. 24, the adaptive flight planning methodology2460 enables anomaly detection, heuristic ranking, automated pathplanning, and faster data acquisition of higher quality, among otherfeatures.

In one or more embodiments, the first and second stages of the imageacquisition can occur in a single flight. For instance, the same UAVduring a single flight can acquire images along a first high-altitudepath, analyze the images to determine the regions of interest, obtainuser feedback regarding the images, generate a second flight path forthe regions of interest, and capture high-resolution images of theregions of interest along the second path prior to landing.

In one or more embodiments, multiple unmanned vehicles (e.g., aircraftand/or ground vehicles) can be utilized for performing the exemplarymethod. The multiple unmanned vehicles can be the same (e.g., having thesame capabilities for one or more of flight, image capture, dataacquisition, communication, processing resources and so forth) or can bedifferent (e.g., having different capabilities for one or more of flight(speed and/or altitude), image capture, data acquisition, communication,processing resources and so forth).

In one embodiment, a high-flying UAV can be utilized for high-altitudeimage capturing and one or more low-flying UAVs can be used forlow-altitude image capturing. In this example, some or each of the high-and low-altitude UAVs can have different image capturing devices suitedfor their particular altitude(s). In one embodiment, the UAVs cancommunicate with each other to provide the second flight plan. Forexample, the high-flying UAV can be utilized for high-altitude imagecapturing and processing of the images to generate a second flight plan.The second flight plan can be wirelessly communicated from thehigh-flying UAV to the one or more low-flying UAVs which can then followthe second flight plan (or a portion thereof if multiple low-flying UAVsare being utilized) to capture low-altitude images of the regions ofinterest. In one or more embodiments, the image processing, usercommunication and/or flight path generation can be performed by a singleUAV (e.g., the high-flying UAV captures/processes high altitude images,obtains user feedback, generates/communicates a second flight plan tolow-flying UAVs, receives high-resolution images from the low-flyingUAVs, and performs any additional image processing and obtaining of userfeedback) or can be performed by more than one of the UAVs (e.g., thehigh-flying UAV captures/processes high altitude images, obtains userfeedback, generates/communicates a second flight plan to one or morelow-flying UAVs, and then the low-flying UAV captures high-resolutionimages along the second flight plan and performs any additional imageprocessing and obtaining of user feedback).

In one or more embodiments, multiple flight plans can be automaticallygenerated to be flown by the same UAV or different UAVs. For example, ahigh-flying UAV can capture/process high altitude images, obtain userfeedback, and generate/communicate a second flight plan to amedium-flying UAV. The medium-flying UAV can capture/process mediumaltitude images along the second path, obtain user feedback, andgenerate/communicate a third flight plan to a low-flying UAV. Thelow-flying UAV can capture/process low altitude images along the thirdflight path, and provide any addition image processing/obtaining userfeedback. Any number of flight paths at any number of altitudesperformed by any number of UAVs can be done in this embodiment. In thisexample, the different sets of images can have different parameters,such as different resolution. In this example, the different flightplans can be utilized for different types of data collection, includingdifferent types of images such as IR images and so forth. In thisexample, some or all of the different flight plans can be at differentaltitudes or can be at the same altitude. For instance, a fourth flightplan can be generated according to regions of interest determined fromprocessing the high-resolution images by the low-flying UAV. The fourthflight plan can be at the same altitude as the third flight plan bututilizes different data acquisition devices (e.g., different types ofcameras), such as capturing color images along the third flight plan andcapturing IR images along the fourth flight plan.

In one or more embodiments, the multiple unmanned vehicles performingthe method can include a UAV and an autonomous ground vehicle. Forexample, a high-flying UAV can capture/process high altitude images,obtain user feedback, and generate/communicate a second flight plan to alow-flying UAV. The low-flying UAV can capture/process medium altitudeimages along the second path, obtain user feedback, andgenerate/communicate a third path to a ground vehicle. The groundvehicle can capture/process ground images or other data along the thirdpath, and provide any addition image processing/obtaining user feedback.In one embodiment, the third path can be a flight plan that is convertedto a ground path, such as by one of the UAVs and/or by the groundvehicle.

In one or more embodiments, different flight plans or paths can begenerated that provide for capturing images of the same regions ofinterest by different vehicle (e.g., UAVs and/or autonomous groundvehicles). In this embodiment, the different flight plans or paths willenable multiple sets of images for the same region of interest(s) toprovide for a more robust data capture process.

In one or more embodiments, the unmanned vehicle(s) (e.g., UAVs and/orautonomous ground vehicles) can be used in conjunction with a mannedvehicle to perform the method, such as a high-altitude flight forhigh-altitude image capturing being flown by a manned aircraft andlow-altitude flights for low-altitude image capturing being flown by anunmanned aircraft. In one or more embodiments, the image processingand/or flight plan generation can be performed in a distributedenvironment which may be limited to the UAVs or be performed in whole orin part by other computing devices, such as a tablet of the user that isproviding feedback for the captured images. For example, captured imagescan be wirelessly transmitted to the tablet so that the tablet cananalyze the images, determine regions of interest and generate a secondflight plan.

In one or more embodiments, different techniques can be utilized fordetermining regions of interest including techniques that performdifferent analysis of the images and/or techniques that analyzedifferent types of data.

In one or more embodiments, a system for monitoring unstructuredenvironments is provided. The system includes a base station comprisinga first central processing unit (CPU) and configured to plan andtransmit a flight path of one or more unmanned aerial vehicles (UAV) asthe one or more UAVs fly along the flight path above a predeterminedarea. The system can use a second CPU of the UAV to capture a pluralityof initial images of the area to gain complete coverage of the areaunder observation at a lower spatial resolution as the UAV flies alongthe flight path and transmitting the plurality of initial images to thefirst CPU. The system can use the first CPU to transform the images fromRGB colorspace to alternative colorspace and analyze the images toautomatically identify contiguous areas within the images that deviatefrom the average of the images that represents anomalous places withinthe predetermined area to identify one or more agronomically anomaloustarget areas. The system can use the first CPU to assign precisegeolocations to the one or more agronomically anomalous target areas andtransmit an updated flight path to the second CPU of the UAV directingthe UAV to the one or more agronomically anomalous target areas. Thesystem can use the second CPU of the UAV, in response to receiving theidentification of one or more anomalous target areas, to automaticallycause the UAV to capture one or more additional high resolution imagesof the one or more target areas. The system can transmit the pluralityof initial images and the one or more additional high resolution imagesto the first CPU.

The system can display high resolution images that overlay at the rightgeolocation to the viewer. The high resolution images can be overlayedat their appropriate geolocation and displayed to a user. The first CPUcan execute an application performing these functions. The system cancollect feedback from the user. The feedback from the user can beselected from text, audio, thumbs up or thumbs down and combinationthereof. The feedback can be annotations related to the image includingpotential causes of the anomalies such as textual annotations or audiorecordings.

The system can execute training of machine learning algorithms toautomatically identify specific causes of anomalies from visual andother contextual information. The contextual information can be selectedfrom time, location, and crop field history. The flight path can bemodified and/or updated dynamically while the UAV is in flight.

The data captured by the second CPU of the UAV can be captured,transmitted and displayed in real time to the first CPU. The pluralityof initial images can be collected by a camera designed to recordelectromagnetic radiation of about 300 nm to about 700 nm. The systemcan include a multi-spectral or hyper-spectral imager.

The plurality of initial images collected can have about a 5% to about a20% area overlap. The plurality of initial images collected can haveabout a 10% area overlap.

The plurality of initial images of the area have a resolution about 5cm/pixel to about 20 cm/pixel. The alternative colorspace can beselected from hue-saturation-value (HSV) and multispec sensors. Thealternative colorspace image can be examined to identify locations thathave different spectral signatures from the average. A grid search ofblocks of images is performed on the alternative colorspace image. Thegrid search of the blocks of images can occur in about a 100×100 matrix.

The spectral signatures of the plurality of initial images can beanalyzed. The first CPU can identify and classify clusters of imagepixels by distinction and form multiple groupings. The first CPU canplan and alter the flight path based on the remaining battery life ofthe UAV and the anomalous index of potential target areas. One or moreadditional high resolution images can be in a range of about 0.1cm/pixel to about 2 cm/pixel.

In one or more embodiments, a processing system (including at least oneprocessor) can obtain a plurality of first images of an area that iscaptured by a camera of a UAV as the UAV flies over the area, whereinthe plurality of first images is at a first spatial resolution. Theprocessing system can analyze at least one initial image of theplurality of first images to automatically identify a target area thatdeviates from a determination of an average of the plurality of firstimages that represents an anomalous place within the area to identifyone or more agronomically anomalous target areas. The processing systemcan assign geolocations to the one or more agronomically anomaloustarget areas. The processing system can generate an updated flight pathaccording to the assigning of the geolocations, wherein the updatedflight path directs the UAV to the one or more agronomically anomaloustarget areas. The processing system can obtain at least one second imageof the one or more agronomically anomalous target areas that is capturedby the camera of the UAV as the UAV flies along the updated flight path,where the at least one second image is at a second spatial resolutionwhich is higher than the first spatial resolution. The processing systemcan be located on the UAV. The processing system can be located on amobile device that is in wireless communication with the UAV, where theprocessing system wirelessly transmits the updated flight path to theUAV, where the plurality of first images and the at least one secondimage are captured by the camera of the UAV during a single flight ofthe UAV, and where the obtaining of the plurality of first images andthe at least one second image by the processing system is via wirelesscommunications with the UAV. The analyzing can include transforming theat least one of the plurality of first images from RGB colorspace toalternative colorspace. The alternative colorspace can be selected fromhue-saturation-value (HSV) or multispectral. The processing system canperform a grid search of blocks of images in the alternative colorspace.

In one or more embodiments a method can include obtaining, by aprocessing system including at least one processor of an unmannedvehicle, a predetermined path, where the predetermined path isdetermined according to an assignment of geolocations to one or moreagronomically anomalous target areas, where the one or moreagronomically anomalous target areas are determined according to ananalysis of a plurality of first images that automatically identifies atarget area that deviates from a determination of an average of theplurality of first images that represents an anomalous place within apredetermined area, where the plurality of first images of thepredetermined area are captured by a camera during a flight over thepredetermined area, and where the plurality of first images is at afirst spatial resolution. The method can include causing, by theprocessing system, the unmanned vehicle to follow the predeterminedpath. The method can include capturing, by the processing system via acamera of the unmanned vehicle, at least one second image of the one ormore agronomically anomalous target areas as the unmanned vehicletravels along the predetermined path, where the at least one secondimage is at a second spatial resolution which is higher than the firstspatial resolution.

The analysis of the plurality of first images can include transformingat least one of the plurality of first images from RGB colorspace toalternative colorspace, where the alternative colorspace is selectedfrom hue-saturation-value (HSV) or multispectral. The unmanned vehiclecan be a ground vehicle, and the plurality of first images of thepredetermined area can be captured by the camera of an unmanned aerialvehicle during the flight over the predetermined area. The unmannedvehicle can be a low-flying unmanned aerial vehicle, and the pluralityof first images of the predetermined area can be captured by the cameraof a high-flying unmanned aerial vehicle during the flight over thepredetermined area. The predetermined path can be determined by a mobiledevice in wireless communication with the processor of the unmannedvehicle, where the one or more agronomically anomalous target areas aredetermined by the mobile device. The at least one second image can beoverlayed at an appropriate geolocation and displayed to a user. Thefeedback can be collected from a user, and the feedback can beassociated with the plurality of first images, the at least one secondimage, or a combination thereof. The feedback from the user can beselected from text, audio, thumbs up or thumbs down, or a combinationthereof. The feedback from the user can include annotations includingpotential causes of anomalies. Machine learning algorithms can betrained to automatically identify specific causes of anomalies.

FIG. 25 depicts an exemplary diagrammatic representation of a machine inthe form of a computer system 2500 within which a set of instructions,when executed, may cause the machine to perform any one or more of themethods described above. One or more instances of the machine canoperate, for example, to perform all or portions of the workflow ofFIGS. 1, 22 and 23 and workflow 2460 of FIG. 24.

In some embodiments, the machine may be connected (e.g., using a network2526) to other machines. In a networked deployment, the machine mayoperate in the capacity of a server or a client user machine in aserver-client user network environment, or as a peer machine in apeer-to-peer (or distributed) network environment.

The machine may comprise a server computer, a client user computer, apersonal computer (PC), a tablet, a smart phone, a laptop computer, adesktop computer, a control system, a network router, switch or bridge,or any machine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. It will beunderstood that a communication device of the subject disclosureincludes broadly any electronic device that provides voice, video ordata communication. Further, while a single machine is illustrated, theterm “machine” shall also be taken to include any collection of machinesthat individually or jointly execute a set (or multiple sets) ofinstructions to perform any one or more of the methods discussed herein.

The computer system 2500 may include a processor (or controller) 2502(e.g., a central processing unit (CPU)), a graphics processing unit(GPU, or both), a main memory 2504 and a static memory 2506, whichcommunicate with each other via a bus 2508. The computer system 2500 mayfurther include a display unit 2510 (e.g., a liquid crystal display(LCD), a flat panel, or a solid state display). The computer system 2500may include an input device 2512 (e.g., a keyboard), a cursor controldevice 2514 (e.g., a mouse), a disk drive unit 2516, a signal generationdevice 2518 (e.g., a speaker or remote control) and a network interfacedevice 2520. In distributed environments, the embodiments described inthe subject disclosure can be adapted to utilize multiple display units2510 controlled by two or more computer systems 2500. In thisconfiguration, presentations described by the subject disclosure may inpart be shown in a first of the display units 2510, while the remainingportion is presented in a second of the display units 2510.

The disk drive unit 2516 may include a tangible computer-readablestorage medium 2522 on which is stored one or more sets of instructions(e.g., software 2524) embodying any one or more of the methods orfunctions described herein, including those methods illustrated above.The instructions 2524 may also reside, completely or at least partially,within the main memory 2504, the static memory 2506, and/or within theprocessor 2502 during execution thereof by the computer system 2500. Themain memory 2504 and the processor 2502 also may constitute tangiblecomputer-readable storage media.

Dedicated hardware implementations including, but not limited to,application specific integrated circuits, programmable logic arrays andother hardware devices can likewise be constructed to implement themethods described herein. Application specific integrated circuits andprogrammable logic array can use downloadable instructions for executingstate machines and/or circuit configurations to implement embodiments ofthe subject disclosure. Applications that may include the apparatus andsystems of various embodiments broadly include a variety of electronicand computer systems. Some embodiments implement functions in two ormore specific interconnected hardware modules or devices with relatedcontrol and data signals communicated between and through the modules,or as portions of an application-specific integrated circuit. Thus, theexample system is applicable to software, firmware, and hardwareimplementations.

In accordance with various embodiments of the subject disclosure, theoperations or methods described herein are intended for operation assoftware programs or instructions running on or executed by a computerprocessor or other computing device, and which may include other formsof instructions manifested as a state machine implemented with logiccomponents in an application specific integrated circuit or fieldprogrammable gate array. Furthermore, software implementations (e.g.,software programs, instructions, etc.) including, but not limited to,distributed processing or component/object distributed processing,parallel processing, or virtual machine processing can also beconstructed to implement the methods described herein. Distributedprocessing environments can include multiple processors in a singlemachine, single processors in multiple machines, and/or multipleprocessors in multiple machines. It is further noted that a computingdevice such as a processor, a controller, a state machine or othersuitable device for executing instructions to perform operations ormethods may perform such operations directly or indirectly by way of oneor more intermediate devices directed by the computing device.

While the tangible computer-readable storage medium 2522 is shown in anexample embodiment to be a single medium, the term “tangiblecomputer-readable storage medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions. The term “tangible computer-readable storage medium” shallalso be taken to include any non-transitory medium that is capable ofstoring or encoding a set of instructions for execution by the machineand that cause the machine to perform any one or more of the methods ofthe subject disclosure. The term “non-transitory” as in a non-transitorycomputer-readable storage includes without limitation memories, drives,devices and anything tangible but not a signal per se.

The term “tangible computer-readable storage medium” shall accordinglybe taken to include, but not be limited to: solid-state memories such asa memory card or other package that houses one or more read-only(non-volatile) memories, random access memories, or other re-writable(volatile) memories, a magneto-optical or optical medium such as a diskor tape, or other tangible media which can be used to store information.Accordingly, the disclosure is considered to include any one or more ofa tangible computer-readable storage medium, as listed herein andincluding art-recognized equivalents and successor media, in which thesoftware implementations herein are stored.

The illustrations of embodiments described herein are intended toprovide a general understanding of the structure of various embodiments,and they are not intended to serve as a complete description of all theelements and features of apparatus and systems that might make use ofthe structures described herein. Many other embodiments will be apparentto those of skill in the art upon reviewing the above description. Theexemplary embodiments can include combinations of features and/or stepsfrom multiple embodiments. Other embodiments may be utilized and derivedtherefrom, such that structural and logical substitutions and changesmay be made without departing from the scope of this disclosure. Figuresare also merely representational and may not be drawn to scale. Certainproportions thereof may be exaggerated, while others may be minimized.Accordingly, the specification and drawings are to be regarded in anillustrative rather than a restrictive sense.

Although specific embodiments have been illustrated and describedherein, it should be appreciated that any arrangement which achieves thesame or similar purpose may be substituted for the embodiments describedor shown by the subject disclosure. The subject disclosure is intendedto cover any and all adaptations or variations of various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, can be used in the subject disclosure.For instance, one or more features from one or more embodiments can becombined with one or more features of one or more other embodiments. Inone or more embodiments, features that are positively recited can alsobe negatively recited and excluded from the embodiment with or withoutreplacement by another structural and/or functional feature. The stepsor functions described with respect to the embodiments of the subjectdisclosure can be performed in any order. The steps or functionsdescribed with respect to the embodiments of the subject disclosure canbe performed alone or in combination with other steps or functions ofthe subject disclosure, as well as from other embodiments or from othersteps that have not been described in the subject disclosure. Further,more than or less than all of the features described with respect to anembodiment can also be utilized.

Less than all of the steps or functions described with respect to theexemplary processes or methods can also be performed in one or more ofthe exemplary embodiments. Further, the use of numerical terms todescribe a device, component, step or function, such as first, second,third, and so forth, is not intended to describe an order or functionunless expressly stated so. The use of the terms first, second, thirdand so forth, is generally to distinguish between devices, components,steps or functions unless expressly stated otherwise. Additionally, oneor more devices or components described with respect to the exemplaryembodiments can facilitate one or more functions, where the facilitating(e.g., facilitating access or facilitating establishing a connection)can include less than every step needed to perform the function or caninclude all of the steps needed to perform the function.

In one or more embodiments, a processor (which can include a controlleror circuit) has been described that performs various functions. Itshould be understood that the processor can be multiple processors,which can include distributed processors or parallel processors in asingle machine or multiple machines. The processor can be used insupporting a virtual processing environment. The virtual processingenvironment may support one or more virtual machines representingcomputers, servers, or other computing devices. In such virtualmachines, components such as microprocessors and storage devices may bevirtualized or logically represented. The processor can include a statemachine, application specific integrated circuit, and/or programmablegate array including a Field PGA. In one or more embodiments, when aprocessor executes instructions to perform “operations”, this caninclude the processor performing the operations directly and/orfacilitating, directing, or cooperating with another device or componentto perform the operations.

FIG. 26A illustrates an embodiment where a first flight path is followedby a UAV 2600 to capture images of a predetermined area (FIG. 26B). Theimages are then processed and analyzed according to one or more of theexemplary embodiments described herein in order to generate the secondflight plan so as to enable capturing of close-up (e.g. high-resolution)images of anomalous areas (FIG. 26C). Various time frames can be usedfor the different flight paths such as a longer time for the firstflight path and a shorter time for the second flight path. In oneembodiment, the first flight path (e.g., high altitude) can be a uniformflight path intended to cover an entire predetermined area whereas thesecond flight path (e.g., low altitude) can be non-uniform intended tocover the specific anomalous area of the entire predetermined area.

REFERENCES

-   [1] Richard Szeliski. Image alignment and stitching: A tutorial.    Foundations and Trends® in Computer Graphics and Vision, 2(1):1-104,    2006.-   [2] S. Leprince, S. Barbot, F. Ayoub, and J. P. Avouac. Automatic    and Precise Orthorectification, Coregistration, and Subpixel    Correlation of Satellite Images, Application to Ground Deformation    Measurements. IEEE Transactions on Geoscience and Remote Sensing,    45(6):1529-1558, June 2007.-   [3] Jorge Torres-S'anchez, Francisca L'opez-Granados, Ana Isabel De    Castro, and Jos'e Manuel Pe{tilde over ( )} na-Barrag'an.    Configuration and specifications of an unmanned aerial vehicle (UAV)    for early site specific weed management. PLoS One, 8(3):e58210,    2013.-   [4] C. Rey, M. P. Mart'in, A. Lobo, I. Luna, M. P. Diago, B. Millan,    and J. Tard'aguila. Multispectral imagery acquired from a UAV to    assess the spatial variability of a Tempranillo vineyard. In John V.    Stafford, editor, Precision agriculture '13, pages 617-624.    Wageningen Academic Publishers, 2013. DOI: 10.3920/978-90-8686-778-3    76.-   [5] Chenghai Yang. Hyperspectral Imagery for Mapping Crop Yield for    Precision Agriculture. In Bosoon Park and Renfu Lu, editors,    Hyperspectral Imaging Technology in Food and Agriculture, Food    Engineering Series, pages 289-304. Springer New York, 2015. DOI:    10.1007/978-1-4939-2836-1 12.-   [6] Won Suk Lee. Plant Health Detection and Monitoring. In Bosoon    Park and Renfu Lu, editors, Hyperspectral Imaging Technology in Food    and Agriculture, Food Engineering Series, pages 275-288. Springer    New York, 2015. DOI: 10.1007/978-1-4939-2836-1 11.-   [7] D. Moshou, D. Kateris, X.-E. Pantazi, and I. Gravalos. Crop and    weed species recognition based on hyperspectral sensing and active    learning. In John V. Stafford, editor, Precision agriculture '13,    pages 555-561. Wageningen Academic Publishers, 2013. DOI:    10.3920/978-90-8686-778-3 68.-   [8] M. Rinaldi, J. Llorens, and E. Gil. Electronic characterization    of the phenological stages of grapevine using a LIDAR sensor. In    John V. Stafford, editor, Precision agriculture '13, pages 603-609.    Wageningen Academic Publishers, 2013. DOI: 10.3920/978-90-8686-778-3    74.-   [9] D. And' ujar, H. Moreno, C. Valero, R. Gerhards, and H. W.    Griepentrog. Weed-crop discrimination using LiDAR measurements. In    John V. Stafford, editor, Precision agriculture' 13, pages 541-545.    Wageningen Academic Publishers, 2013. DOI: 10.3920/978-90-8686-778-3    66.-   [10] A. Matese, F. Capraro, J. Primicerio, G. Gualato, S. F. Di    Gennaro, and G. Agati. Mapping of vine vigor by UAV and anthocyanin    content by a non-destructive fluorescence technique. In Precision    agriculture' 13, pages 201-208. Springer, 2013.-   [11] M. Teena and A. Manickavasagan. Thermal Infrared Imaging. In    Annamalai Manickavasagan and Hemantha Jayasuriya, editors, Imaging    with Electromagnetic Spectrum, pages 147-173. Springer Berlin    Heidelberg, 2014. DOI: 10.1007/978-3-642-54888-8 8.-   [12] M. Meron, V. Alchanatis, Y. Cohen, and J. Tsipris. Aerial    thermography for crop stress evaluation—a look into the state of the    technology. In Precision agriculture' 13, pages 177-183. Springer,    2013.-   [13] P. J. Curran and M. D. Steven. Multispectral Remote Sensing for    the Estimation of Green Leaf Area Index [and Discussion].    Philosophical Transactions of the Royal Society of London A:    Mathematical, Physical and Engineering Sciences, 309(1508):257-270,    July 1983.-   [14] R. S. Defries and J. R. G. Townshend. NDVI-derived land cover    classifications at a global scale. International Journal of Remote    Sensing, 15(17):3567-3586, November 1994.-   [15] Joe Mari J. Maja, Todd Campbell, Joao Camargo Neto, and Philip    Astillo. Predicting cotton yield of small field plots in a cotton    breeding program using UAV imagery data. In SPIE Commercial+    Scientific Sensing and Imaging, pages 98660C-98660C. International    Society for Optics and Photonics, 2016.-   [16] Ryo Sugiura, Shogo Tsuda, Seiji Tamiya, Atsushi Itoh, Kentaro    Nishiwaki, Noriyuki Murakami, Yukinori Shibuya, Masayuki Hirafuji,    and Stephen Nuske. Field phenotyping system for the assessment of    potato late blight resistance using RGB imagery from an unmanned    aerial vehicle. Biosystems Engineering, 148:1-10, 2016.-   [17] D. Zermas, D. Teng, P. Stanitsas, M. Bazakos, D. Kaiser, V.    Morellas, D. Mulla, and N. Papanikolopoulos. Automation solutions    for the evaluation of plant health in corn fields. In 2015 IEEE/RSJ    International Conference on Intelligent Robots and Systems (IROS),    pages 6521-6527, September 2015.-   [18] Pauli J. Komi, Mike R. Jackson, and Rob M. Parkin. Plant    classification combining colour and spectral cameras for weed    control purposes. In 2007 IEEE International Symposium on Industrial    Electronics, pages 2039-2042. IEEE, 2007.-   [19] Lorenz Meier, J F Camacho, B Godbolt, J Goppert, L Heng, M    Lizarraga, et al. Mavlink: Micro air vehicle communication protocol.    Online]. Tillg {umlaut over ( )} anglig:    http://qgroundcontrol.org/mavlink/start.[H {umlaut over ( )}amtad    2014-05-22], 2013.-   [20] R. C. Grande, T. J. Walsh, G. Chowdhary, S. Ferguson, and J. P.    How. Online regression for data with changepoints using gaussian    processes and reusable models. IEEE Transactions on Neural Networks    and Learning Systems, PP(99):1-14, 2016.-   [21] Brian Kulis and Michael I Jordan. Revisiting k-means: New    algorithms via bayesian nonparametrics. In Proceedings of the 29th    International Conference on Machine Learning (ICML-12), pages    513-520, 2012.-   [22] Isabelle Philipp and Thomas Rath. Improving plant    discrimination in image processing by use of different colour space    transformations. Computers and electronics in agriculture,    35(1):1-15, 2002.-   [23] Jonas Fernandes da Silva, Alisson V Brito, Helder Nogueira de    Moura, et al. An embedded system for aerial image processing from    unmanned aerial vehicles. In 2015 Brazilian Symposium on Computing    Systems Engineering (SBESC), pages 154-157. IEEE, 2015.-   [24] Luis F Gonzalez, Glen A Montes, Eduard Puig, Sandra Johnson,    Kerrie Mengersen, and Kevin J Gaston. Unmanned aerial vehicles    (uavs) and artificial intelligence revolutionizing wildlife    monitoring and conservation. Sensors, 16(1):97, 2016.-   [25] Jacopo Primicerio, Salvatore Filippo Di Gennaro, Edoardo    Fiorillo, Lorenzo Genesio, Emanuele Lugato, Alessandro Matese, and    Francesco Primo Vaccari. A flexible unmanned aerial vehicle for    precision agriculture. Precision Agriculture, 13(4):517-523, 2012.-   [26] M Govender, K Chetty, and H Bulcock. A review of hyperspectral    remote sensing and its application in vegetation and water resource    studies. Water Sa, 33(2), 2007.-   [27] George Alan Blackburn. Hyperspectral remote sensing of plant    pigments. Journal of experimental botany, 58(4):855-867, 2007.-   [28] Allan Axelrod, Girish Chowdhary, and Sertac Karaman.    Efficiently informed exploration between isolated operatives. In    Conference on Decisions and Control, Osaka, Japan, 2015. IEEE.    Submitted.-   [29] Allan Axelrod and Girish Chowdhary. Uninformed-to-informed    exploration in unstructured real-world environments. In    Self-Confidence In Autonomous Systems, Washington D.C., USA, 2015.    AAAI.-   [30] Allan Axelrod and Girish Chowdhary. The Explore-Exploit Dilemma    in Nonstationary Decision Making under Uncertainty, chapter The    Explore-Exploit Dilemma in Nonstationary Decision Making under    Uncertainty. 2198-4182. Springer international publishing, 1    edition, 2015.-   [31] Nazim Kemal Ure, Girish Chowdhary, Jonathan P. How, Mathew    Vavarina, and John Vian. Health aware planning under uncertainty for    uav missions with heterogeneous teams. In Proceedings of the    European Control Conference, Zurich, Switzerland, July 2013.-   [32] Allan Axelrod and Girish Chowdhary. Adaptive algorithms for    autonomous dataferrying in nonstationary environments. In Aerospace    Science and Technology Forum, Kissimmee, Fla., 2015. AIAA.-   [33] Rakshit Allamaraju, Hassan Kingravi, Allan Axelrod, Girish    Chowdhary, Robert Grande, Christopher Crick, Weihua Sheng, and    Jonathan How. Human aware path planning in urban environments with    nonstationary mdps. In International Conference on Robotics and    Automation, Hong Kong, China, 2014. IEEE.-   [34] Michael R Gary and David S Johnson. Computers and    intractability: A guide to the theory of np-completeness, 1979.-   [35] K E Trummel and J R Weisinger. Technical note—the complexity of    the optimal searcher path problem. Operations Research,    34(2):324-327, 1986.-   [36] Kaarthik Sundar and Sivakumar Rathinam. Algorithms for routing    an unmanned aerial vehicle in the presence of refueling depots. IEEE    Transactions on Automation Science and Engineering, 11(1):287-294,    2014.-   [37] Alborz Geramifard, Thomas J. Walsh, Stefanie Tellex, Girish    Chowdhary, Nicholas Roy, and Jonathan P. How. A tutorial on linear    function approximators for dynamic programming and reinforcement    learning. Foundations and Trends® in Machine Learning, 6(4):375-451,    2013.-   [38] Girish Chowdhary, Miao Liu, Robert C. Grande, Thomas J. Walsh,    Jonathan P. How, and Lawrence Carin. Off-Policy Reinforcement    Learning with Gaussian Processes. Acta Automatic Sinica (to appear),    2014.-   [39] N. Kemal Ure, Alborz Geramifard, Girish Chowdhary, and    Jonathan P. How. Adaptive Planning for Markov Decision Processes    with Uncertain Transition Models via Incremental Feature Dependency    Discovery. In European Conference on Machine Learning (ECML), 2012.-   [40] Trevor Campbell, Sameera S. Ponda, Girish Chowdhary, and    Jonathan P. How. Planning under uncertainty using nonparametric    bayesian models. In AIAA Guidance, Navigation, and Control    Conference (GNC), August 2012.-   [41] G. Chowdhary, H. A. Kingravi, J. P. How, and P. A. Vela.    Bayesian nonparametric adaptive control using gaussian processes.    Neural Networks and Learning Systems, IEEE Transactions on,    PP(99):1-1, 2014.-   [42] Hassan Kingravi, Harshal Maske, and Girish Chowdhary. Kernel    observers: Systems theoretic modeling and inference of    spatiotemporally varying processes. In Advances in Neural    Information Processing Systems (NIPS), Barcelona, Spain, 2016.    accepted.-   [43] H. A. Kingravi, H. Maske, and G. Chowdhary. Kernel controllers:    A systems-theoretic approach for data-driven modeling and control of    spatiotemporally evolving processes. In 2015 54th IEEE Conference on    Decision and Control (CDC), pages 7365-7370, December 2015.-   [44] Rob Leclerc. Agtech investing report. Technical report,    AgFunder, August 2016.

The invention claimed is:
 1. A system for monitoring environments, thesystem comprising: a base station comprising a first processing systemincluding at least one first processor; and an unmanned aerial vehicle(UAV) comprising a second processing system including at least onesecond processor, wherein the UAV is in communication with the basestation, wherein the second processing system of the UAV is configuredto capture a plurality of initial images of a predetermined area toobtain coverage of the predetermined area at a lower spatial resolutionas the UAV flies along a flight path, wherein the flight path for theUAV is above the predetermined area, and wherein the second processingsystem of the UAV is configured to transmit the plurality of initialimages to the first processing system of the base station, wherein thefirst processing system of the base station is configured to analyze theat least one of the plurality of initial images to automaticallyidentify areas within the at least one of the plurality of initialimages that deviate from a determination of an average of the pluralityof initial images and that represent anomalous places within thepredetermined area to identify a plurality of agronomically anomaloustarget areas, and wherein the plurality of agronomically anomaloustarget areas together comprise a set of agronomically anomalous targetareas, wherein the first processing system of the base station isconfigured to assign geolocations to each of the agronomically anomaloustarget areas of the set, wherein the first processing system of the basestation is configured to select one or more of the agronomicallyanomalous target areas of the set to form a subset of one or moreagronomically anomalous target areas, wherein a number of agronomicallyanomalous target areas in the set is larger than is in the subset,wherein each agronomically anomalous target area in the subset isselected by the first processing system of the base station as a resultof that agronomically anomalous target area being larger than at leastone agronomically anomalous target area of the set that is excluded fromthe subset, and wherein the first processing system of the base stationis configured to transmit an updated flight path to the secondprocessing system of the UAV directing the UAV to the one or moreagronomically anomalous target areas of the subset, wherein the secondprocessing system of the UAV is configured to, in response to receivingidentification of the one or more agronomically anomalous target areasof the subset, automatically cause the UAV to capture one or moreadditional high resolution images of the one or more agronomicallyanomalous target areas of the subset, and wherein the second processingsystem of the UAV is configured to transmit the one or more additionalhigh resolution images to the first processing system of the basestation.
 2. The system of claim 1, wherein the first processing systemof the base station is configured to examine the at least one of theplurality of initial images that has been transformed to an alternativecolorspace to identify locations that have different spectral signaturesfrom the average.
 3. The system of claim 2, wherein the at least one ofthe plurality of initial images has been transformed from an RGBcolorspace to the alternative colorspace, and wherein the alternativecolorspace is selected from hue-saturation-value (HSV) or multispectral.4. The system of claim 3, wherein the first processing system of thebase station is configured to perform a grid search of blocks of imagesin the alternative colorspace.
 5. A non-transitory computer-readablestorage medium comprising executable instructions that, when executed bya processing system including at least one processor, perform operationscomprising: analyzing at least one initial image of a plurality of firstimages of an area to automatically identify a plurality of target areas,each of which deviates from a determination of an average of theplurality of first images and each of which represents an anomalousplace within the area that is identified as an agronomically anomaloustarget area, the agronomically anomalous target areas collectivelyforming a set of agronomically anomalous target areas, wherein theplurality of first images of the area are captured by a camera of anunmanned aerial vehicle (UAV) as the UAV flies over the area, andwherein the plurality of first images is at a first spatial resolution;selecting one or more of the agronomically anomalous target areas of theset to form a subset of one or more agronomically anomalous targetareas, wherein a number of agronomically anomalous target areas in theset is larger than is in the subset, wherein each agronomicallyanomalous target area in the subset is selected as a result of thatagronomically anomalous target area being larger than at least oneagronomically anomalous target area of the set that is excluded from thesubset; generating an updated flight path according to the selecting ofthe subset, wherein the updated flight path directs the UAV to the oneor more agronomically anomalous target areas of the subset; andobtaining at least one second image of the one or more agronomicallyanomalous target areas of the subset that is captured by the camera ofthe UAV as the UAV flies along the updated flight path, wherein the atleast one second image is at a second spatial resolution which is higherthan the first spatial resolution.
 6. The non-transitorycomputer-readable storage medium of claim 5, wherein the processingsystem is located on the UAV.
 7. The non-transitory computer-readablestorage medium of claim 5, wherein the processing system is located on amobile device that is in wireless communication with the UAV, whereinthe processing system wirelessly transmits the updated flight path tothe UAV, wherein the plurality of first images and the at least onesecond image are captured by the camera of the UAV during a singleflight of the UAV, and wherein the obtaining of the plurality of firstimages and the at least one second image by the processing system is viawireless communications with the UAV.
 8. The non-transitorycomputer-readable storage medium of claim 5, wherein the analyzingcomprises transforming the at least one initial image of the pluralityof first images from RGB colorspace to alternative colorspace.
 9. Thenon-transitory computer-readable storage medium of claim 8, wherein thealternative colorspace is selected from hue-saturation-value (HSV) ormultispectral.
 10. The non-transitory computer-readable storage mediumof claim 8, wherein the operations further comprise performing a gridsearch of blocks of images in the alternative colorspace.
 11. A methodcomprising: obtaining, by a processing system including at least oneprocessor of an unmanned vehicle, a predetermined path, wherein thepredetermined path is determined according to: an analysis of at leastone initial image of a plurality of first images to automaticallyidentify a plurality of target areas, each of which deviates from adetermination of an average of the plurality of first images and each ofwhich identifies an agronomically anomalous target area, wherein theagronomically anomalous target areas collectively form a set ofagronomically anomalous target areas, wherein the plurality of firstimages are captured by a camera during a flight over a predeterminedarea, and wherein each of the plurality of first images is at a firstspatial resolution; and a selection of one or more of the agronomicallyanomalous target areas of the set to form a subset of one or moreagronomically anomalous target areas, wherein a number of agronomicallyanomalous target areas in the set is larger than is in the subset,wherein each agronomically anomalous target area in the subset isselected as a result of that agronomically anomalous target area beinglarger than at least one agronomically anomalous target area of the setthat is excluded from the subset; and capturing, by the processingsystem via a camera of the unmanned vehicle, at least one second imageof the one or more agronomically anomalous target areas of the subset asthe unmanned vehicle is caused to travel along the predetermined path,wherein the at least one second image is at a second spatial resolutionwhich is higher than the first spatial resolution.
 12. The method ofclaim 11, wherein the analysis of the at least one initial imagecomprises transforming the at least one initial image from RGBcolorspace to alternative colorspace, and wherein the alternativecolorspace is selected from hue-saturation-value (HSV) or multispectral.13. The method of claim 11, wherein the unmanned vehicle is a groundvehicle, the camera is part of an unmanned aerial vehicle, and theplurality of first images of the predetermined area are captured by thecamera of the unmanned aerial vehicle during the flight over thepredetermined area.
 14. The method of claim 11, wherein the unmannedvehicle is a low-flying unmanned aerial vehicle, the camera is part of ahigh-flying unmanned aerial vehicle, and the plurality of first imagesof the predetermined area are captured by the camera of the high-flyingunmanned aerial vehicle during the flight over the predetermined area.15. The method of claim 11, wherein the predetermined path is determinedby a mobile device in wireless communication with the processor of theunmanned vehicle, and wherein the one or more agronomically anomaloustarget areas of the subset are determined by the mobile device.
 16. Themethod of claim 11, wherein the at least one second image is overlayedat an appropriate geolocation and displayed to a user.
 17. The method ofclaim 11, wherein feedback is collected from a user, and wherein thefeedback is associated with the plurality of first images, the at leastone second image, or a combination thereof.
 18. The method of claim 17,wherein the feedback from the user is selected from text, audio, thumbsup or thumbs down, or a combination thereof.
 19. The method of claim 17,wherein the feedback from the user comprises annotations includingpotential causes of anomalies.
 20. The method of claim 11, whereinmachine learning algorithms are trained to automatically identifyspecific causes of anomalies.